Add SGL support for IB rc, cuda copy and cuda IPC (imported)#11637
Open
bosilca wants to merge 9 commits into
Open
Add SGL support for IB rc, cuda copy and cuda IPC (imported)#11637bosilca wants to merge 9 commits into
bosilca wants to merge 9 commits into
Conversation
f919e9e to
a7edbe4
Compare
Downgrade -Werror=unknown-pragmas to a warning so compilers that do not support OpenMP '#pragma omp' can still build the tree.
Add the shared RC foundation for scatter-gather put_sgl_zcopy: advertise the maximum number of segments (bounded by the QP depth), expose a capability query helper, and record the required TX credit count so a multi-WQE SGL operation is queued as pending instead of busy-spinning when credits are temporarily unavailable.
Implement uct_rc_verbs_ep_put_sgl_zcopy, posting the whole segment list as a chained batch of RDMA_WRITE work requests with a single doorbell ring, and wire the operation and its v2 capability into the interface.
Implement uct_rc_mlx5_base_ep_put_sgl_zcopy, posting one single-SGE RDMA_WRITE WQE per segment as a batch. Add an iov_post_flags argument to uct_rc_mlx5_txqp_dptr_post_iov to skip forced TX moderation on the inner batched WQEs, and wire the operation and its v2 capability into the interface. The dc_mlx5 caller is updated for the new post_iov signature.
Implement uct_cuda_copy_ep_put_sgl_zcopy for intra-process device copies and use a process-wide interface id so cuda_copy interfaces within the same process are mutually reachable and can serve as the SGL RMA lane.
Implement uct_cuda_ipc_ep_put_sgl_zcopy for inter-process device memory transfers over the SGL put API. This code originates from michal-shalev's uct-cuda-ipc-sgl branch.
Enable the UCP SGL put tests and adapt the CUDA IPC device SGL test to the put_sgl_zcopy API (counts/strides), moving capability checks after entity initialization and skipping when CUDA memory detection is unavailable.
Add a '-D sgl[/N]' datatype to ucx_perftest that splits the message into N equal-length segments and drives ucp_put_bw through the SGL put API. The SGL state is heap-allocated to avoid inflating the runner stack frame, and the option is restricted to the UCP put bandwidth path.
Signed-off-by: George Bosilca <gbosilca@nvidia.com>
a7edbe4 to
4d03b64
Compare
Contributor
|
Hi George, thanks for posting this PR. |
brminich
requested changes
Jul 9, 2026
brminich
left a comment
Contributor
There was a problem hiding this comment.
conflicts with other SGL Prs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds native UCT transport support for the scatter-gather list (SGL) zero-copy put operation (uct_ep_put_sgl_zcopy), which previously only had the UCP software-emulation path. This lets a multi-segment ucp_put_nbx SGL operation be offloaded directly to the transport as a single batched submission instead of being decomposed into individual puts.
Implemented for:
Also adds gtest coverage and a ucx_perftest bandwidth mode for benchmarking.
Details / commits
UCT/IB/RC (base): advertise the max SGL segment count (bounded by QP depth), add a v2 capability-query helper, and reserve the full TX credit count for a multi-WQE SGL op so it is queued as pending instead of busy-spinning when credits are momentarily short.
Notes
Tests
Follow-up
uct_ib_md_direct_rkey()calls in the RC put paths. Both the mlx5 and verbs put flows already resolve the remote key in place via their fence_put helpers —uct_rc_mlx5_ep_fence_put()anduct_rc_ep_fence_put()set*rkey = uct_ib_md_direct_rkey(*rkey)(or the atomic-resolve variant) before the key is ever posted. As a result, (1) theuct_ib_md_direct_rkey(rdma_rkey)insideuct_rc_mlx5_base_ep_zcopy_post()(rc_mlx5.inl) and (2) theuct_ib_md_direct_rkey(rkey)in the verbs SGL fill (UCT_RC_VERBS_FILL_RDMA_WR inuct_rc_verbs_ep_put_sgl_zcopy) both re-apply the conversion to an already-resolved key. Sinceuct_ib_md_direct_rkey()is just a low-32-bit mask ((uint32_t)uct_rkey), re-applying it to a value whose high bits are already cleared is idempotent — a no-op — so these calls are harmless but redundant. (On mlx5 it's doubly redundant, becauseuct_ib_mlx5_ep_set_rdma_seg()only writes the low 32 bits of the key into the WQE regardless.) They can be dropped for clarity, or kept for visual symmetry; either way there is no behavioral difference.